Inter-observer variation in the diagnosis of coronal articular fracture lines in the lunate facet of the distal radius

Background: Several studies support the use of CT for diagnosing coronal fractures of the distal radius but the inter-observer reliability of these observations is less well studied. We tested the null hypothesis that radiographs alone and the combination of radiographs and two-dimensional computed tomography scans (2DCT) have the same inter-observer variation for the diagnosis of coronal articular fracture lines in the distal radius. Methods: Using a web-based survey, 63 surgeons were randomized to evaluate 16 fractures of the distal radius on radiographs alone or radiographs and 2DCT for the presence or absence of a coronal fracture line of the lunate facet and, if present, the stability of the fracture. The kappa multirater measure was calculated to estimate agreement between observers. Results: The inter-observer variation in diagnosis of a coronal fracture line was fair with both radiographs and 2DCT, as was the diagnosis of instability of the volar lunate facet fracture when present. Conclusion: Two-dimensional computed tomography does not improve observer agreement on the diagnosis of coronal plane articular fracture lines in the lunate facet of the distal radius. © 2012 American Association for Hand Surgery.

Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study

OBJECTIVE: To study the inter-observer variation related to extraction of continuous and numerical rating scale data from trial reports for use in meta-analyses. DESIGN: Observer agreement study. DATA SOURCES: A random sample of 10 Cochrane reviews that presented a result as a standardised mean difference (SMD), the protocols for the reviews and the trial reports (n=45) were retrieved. DATA EXTRACTION: Five experienced methodologists and five PhD students independently extracted data from the trial reports for calculation of the first SMD result in each review. The observers did not have access to the reviews but to the protocols, where the relevant outcome was highlighted. The agreement was analysed at both trial and meta-analysis level, pairing the observers in all possible ways (45 pairs, yielding 2025 pairs of trials and 450 pairs of meta-analyses). Agreement was defined as SMDs that differed less than 0.1 in their point estimates or confidence intervals. RESULTS: The agreement was 53% at trial level and 31% at meta-analysis level. Including all pairs, the median disagreement was SMD=0.22 (interquartile range 0.07-0.61). The experts agreed somewhat more than the PhD students at trial level (61% v 46%), but not at meta-analysis level. Important reasons for disagreement were differences in selection of time points, scales, control groups, and type of calculations; whether to include a trial in the meta-analysis; and data extraction errors made by the observers. In 14 out of the 100 SMDs calculated at the meta-analysis level, individual observers reached different conclusions than the originally published review. CONCLUSIONS: Disagreements were common and often larger than the effect of commonly used treatments. Meta-analyses using SMDs are prone to observer variation and should be interpreted with caution. The reliability of meta-analyses might be improved by having more detailed review protocols, more than one observer, and statistical expertise.

Variation in pre-PCR processing of FFPE samples leads to discrepancies in BRAF and EGFR mutation detection: a diagnostic RING trial.

AIMS: Mutation detection accuracy has been described extensively; however, it is surprising that pre-PCR processing of formalin-fixed paraffin-embedded (FFPE) samples has not been systematically assessed in clinical context. We designed a RING trial to (i) investigate pre-PCR variability, (ii) correlate pre-PCR variation with EGFR/BRAF mutation testing accuracy and (iii) investigate causes for observed variation. METHODS: 13 molecular pathology laboratories were recruited. 104 blinded FFPE curls including engineered FFPE curls, cell-negative FFPE curls and control FFPE tissue samples were distributed to participants for pre-PCR processing and mutation detection. Follow-up analysis was performed to assess sample purity, DNA integrity and DNA quantitation. RESULTS: Rate of mutation detection failure was 11.9%. Of these failures, 80% were attributed to pre-PCR error. Significant differences in DNA yields across all samples were seen using analysis of variance (p

bcl-2 and c-erbB-2 proteins are involved in the regulation of VEGF and of thymidine phosphorylase angiogenic activity in non-small-cell lung cancer

Tumour angiogenesis has been recently recognised as one of the most important prognostic factors in lung cancer. Although a variety of angiogenic factors have been identified, the angiogenesis process remains poorly understood. Bcl-2, c-erbB-2 and p53 are well-known oncogenes involved in non- small-cell lung cancer pathogenesis. A direct correlation of thymidine phosphorylase (TP) and of vascular endothelial growth factor (VEGF) with intratumoural angiogenesis has been reported. In the present study we investigated the possible regulatory role if bcl-2, c-erB-2 proteins in angiogenesis and in VEGF and TP expression in non-small-cell lung cancer. Two hundred sixteen specimens from T1,2-NO, 1 staged patients treated with surgery alone were immunohistochemically examined. Bcl-2 and c-erbB-2 were significantly inversely related to each other (P = 0.04) and both were inversely associated with microvessel density (P < 0.02). High TP and VEGF reactivity was statistically related to loss of bcl-2 expression (P < 0.01). A significant co-expression of c-erbB-2 with TP was noted (P = 0.01). However, TP expression was related to high angiogenesis only in cases with absence of c-erB-2 expression (P < 0.0001). c-erbB-2 expression in poorly vascularised tumours was linked with poor outcome (P = 0.03). The present study provides strong evidence that the bcl-2 gene has a suppressive function over genes involved in both angiogenesis (VEGF and TP) and cell migration (c- erbB-2) in NSCLC. TP and c-erbB-2 proteins are significantly, and often simultaneously, expressed in bcl-2 negative cases. However, expression of the c-erbB-2 abolishes the TP-related angiogenic activity. Whether this is a result of a direct activity of the c-erbB-2 protein or a consequence of a c- erbB-2-related immune response remains to be further investigated.

Whole-genome linkage analysis of rheumatoid arthritis susceptibility loci in 252 affected sibling pairs in the United Kingdom

Objective. To undertake a systematic wholegenome screen to identify regions exhibiting genetic linkage to rheumatoid arthritis (RA). Methods. Two hundred fifty-two RA-affected sibling pairs from 182 UK families were genotyped using 365 highly informative microsatellite markers. Microsatellite genotyping was performed using fluorescent polymerase chain reaction primers and semiautomated DNA sequencing technology. Linkage analysis was undertaken using MAPMAKER/SIBS for single-point and multipoint analysis. Results. Significant linkage (maximum logarithm of odds score 4.7 [P = 0.000003] at marker D6S276, 1 cM from HLA-DRB1) was identified around the major histocompatibility complex (MHC) region on chromosome 6. Suggestive linkage (P < 7.4 × 10-4) was identified on chromosome 6q by single- and multipoint analysis. Ten other sites of nominal linkage (P < 0.05) were identified on chromosomes 3p, 4q, 7p, 2 regions of 10q, 2 regions of 14q, 16p, 21q, and Xq by single-point analysis and on 3 sites (1q, 14q, and 14q) by multipoint analysis. Conclusion. Linkage to the MHC region was confirmed. Eleven non-HLA regions demonstrated evidence of suggestive or nominal linkage, but none reached the genome-wide threshold for significant linkage (P = 2.2 × 10-5). Results of previous genome screens have suggested that 6 of these regions may be involved in RA susceptibility.

Automatic segmentation of seven retinal layers in SDOCT images congruent with expert manual segmentation.

Segmentation of anatomical and pathological structures in ophthalmic images is crucial for the diagnosis and study of ocular diseases. However, manual segmentation is often a time-consuming and subjective process. This paper presents an automatic approach for segmenting retinal layers in Spectral Domain Optical Coherence Tomography images using graph theory and dynamic programming. Results show that this method accurately segments eight retinal layer boundaries in normal adult eyes more closely to an expert grader as compared to a second expert grader.

An automated machine vision system for the histological grading of cervical intraepithelial neoplasia (CIN)

The histological grading of cervical intraepithelial neoplasia (CIN) remains subjective, resulting in inter- and intra-observer variation and poor reproducibility in the grading of cervical lesions. This study has attempted to develop an objective grading system using automated machine vision. The architectural features of cervical squamous epithelium are quantitatively analysed using a combination of computerized digital image processing and Delaunay triangulation analysis; 230 images digitally captured from cases previously classified by a gynaecological pathologist included normal cervical squamous epithelium (n = 30), koilocytosis (n = 46), CIN 1 (n = 52), CIN 2 (n = 56), and CIN 3 (n=46). Intra- and inter-observer variation had kappa values of 0.502 and 0.415, respectively. A machine vision system was developed in KS400 macro programming language to segment and mark the centres of all nuclei within the epithelium. By object-oriented analysis of image components, the positional information of nuclei was used to construct a Delaunay triangulation mesh. Each mesh was analysed to compute triangle dimensions including the mean triangle area, the mean triangle edge length, and the number of triangles per unit area, giving an individual quantitative profile of measurements for each case. Discriminant analysis of the geometric data revealed the significant discriminatory variables from which a classification score was derived. The scoring system distinguished between normal and CIN 3 in 98.7% of cases and between koilocytosis and CIN 1 in 76.5% of cases, but only 62.3% of the CIN cases were classified into the correct group, with the CIN 2 group showing the highest rate of misclassification. Graphical plots of triangulation data demonstrated the continuum of morphological change from normal squamous epithelium to the highest grade of CIN, with overlapping of the groups originally defined by the pathologists. This study shows that automated location of nuclei in cervical biopsies using computerized image analysis is possible. Analysis of positional information enables quantitative evaluation of architectural features in CIN using Delaunay triangulation meshes, which is effective in the objective classification of CIN. This demonstrates the future potential of automated machine vision systems in diagnostic histopathology. Copyright (C) 2000 John Wiley and Sons, Ltd.

Immunohistochemistry should undergo robust validation equivalent to that of molecular diagnostics

Immunohistochemistry (IHC) is a widely available and highly utilised tool in diagnostic histopathology and is used to guide treatment options as well as provide prognostic information. IHC is subjected to qualitative and subjective assessment, which has been criticised for a lack of stringency, while PCR-based molecular diagnostic validations by comparison are regarded as very rigorous. It is essential that IHC tests are validated through evidence-based procedures. With the move to ISO15189 (2012), not just of the accuracy, specificity and reproducibility of each test need to be determined and managed, but also the degree of uncertainty and the delivery of such tests. The recent update to ISO 15189 (2012) states that it is appropriate to consider the potential uncertainty of measurement of the value obtained in the laboratory and how that may impact on prognostic or predictive thresholds. In order to highlight the problems surrounding IHC validity, we reviewed the measurement of Ki67and p53 in the literature. Both of these biomarkers have been incorporated into clinical care by pathology laboratories worldwide. The variation seen appears excessive even when measuring centrally stained slides from the same cases. We therefore propose in this paper to establish the basis on which IHC laboratories can bring the same level of robust validation seen in the molecular pathology laboratories and the principles applied to all routine IHC tests.

Seven years of gamma-ray spectrometry interlaboratory comparisons in Switzerland.

Since the 1990s, regular comparisons of gamma-ray spectrometry in Switzerland were organized to improve laboratory abilities to measure the radioactivity in the environment and food stuffs at typical routine levels. The activity concentration of the test samples and the evaluation of the associated uncertainties remained each year the main required test result. Over the years, the comparisons used certified reference solutions as well as environmental samples. The aim of this study is to research the effect of the comparisons on measurement quality. An analysis of the seven last interlaboratory comparisons revealed that the Swiss measurement capability is up to date. In addition, the results showed that the participants now have an improved evaluation of the uncertainties associated with their measurement.

Variability of ascending aorta diameter measurements as assessed with electrocardiography-gated multidetector computerized tomography and computer assisted diagnosis software.

Recently, morphometric measurements of the ascending aorta have been done with ECG-gated multidector computerized tomography (MDCT) to help the development of future novel transcatheter therapies (TCT); nevertheless, the variability of such measurements remains unknown. Thirty patients referred for ECG-gated CT thoracic angiography were evaluated. Continuous reformations of the ascending aorta, perpendicular to the centerline, were obtained automatically with a commercially available computer aided diagnosis (CAD). Then measurements of the maximal diameter were done with the CAD and manually by two observers (separately). Measurements were repeated one month later. The Bland-Altman method, Spearman coefficients, and a Wilcoxon signed-rank test were used to evaluate the variability, the correlation, and the differences between observers. The interobserver variability for maximal diameter between the two observers was up to 1.2 mm with limits of agreement [-1.5, +0.9] mm; whereas the intraobserver limits were [-1.2, +1.0] mm for the first observer and [-0.8, +0.8] mm for the second observer. The intraobserver CAD variability was 0.8 mm. The correlation was good between observers and the CAD (0.980-0.986); however, significant differences do exist (P<0.001). The maximum variability observed was 1.2 mm and should be considered in reports of measurements of the ascending aorta. The CAD is as reproducible as an experienced reader.

Impacto de un curso teórico de mamografía en la categorización de microcalcificaciones de acuerdo al BI-RADS 4ed

Introducción: Por medio del presente estudio se busca determinar el impacto de un curso teórico de mamografía en la categorización de microcalficaciones mamarias. Materiales y métodos: Se realizó un estudio doble ciego para evaluar la variación de los valores predictivos positivos (VPP) de los descriptores morfológicos y de distribución de calcificaciones del BI-RADS 4ed, luego de un curso teórico de mamografía de 6 horas distribuido en 12 charlas semanales de 30 minutos. Se utilizó la base de datos de un estudio previo, en el cual dos radiólogos clasificaron las calcificaciones presentes en 108 mamografías de pacientes con diagnóstico histológico. Se realizó una nueva categorización de las calcificaciones según los descriptores. Se calcularon además, los valores kappa ponderados de concordancia intra e inter-observador, la sensibilidad y especificidad. Resultados: Para las calcificaciones de alta probabilidad de malignidad (pleomórficas finas y lineales ramificadas en conjunto), se obtuvieron VPP de 41 y 40%, con aumento leve respecto a la lectura inicial. El índice kappa ponderado para el acuerdo inter-observador fue de 0,305 lo que indica concordancia débil, con discreto incremento respecto a la evaluación inicial 0,260. La concordancia intra-observador para el evaluador 1 dio un valor kappa 0,471, moderada; y para el evaluador 2 un valor kappa 0,400, débil. Discusión: Un módulo teórico corto permite mejorar ligeramente la concordancia inter-observador en la evaluación de microcalcificaciones mamarias mediante el BI-RADS 4ed, así como los valores predictivos positivos de microcalficaciones de alta probabilidad de malignidad.

Interobserver reproducibility of histopathologic prognostic variables in primary cutaneous melanomas

Background: The prognosis for patients with localized primary cutaneous melanoma is known to depend principally on tumor thickness, and to a lesser extent on ulcerative state and Clark level. We have recently found in an analysis of 3661 patients that tumor mitotic rate (TMR) is also an important prognostic parameter, ranking second only to tumor thickness. However, few studies have assessed the accuracy and reproducibility with which these features of a melanoma are recorded by histopathologists.
Aim: To assess interobserver reproducibility of major pathologic prognostic parameters in cutaneous melanoma.
Methods: Single hematoxylin and eosin-stained slides of 69 dermally invasive primary cutaneous melanomas were circulated among six pathologists with differing experience in the assessment of melanocytic tumors. The observers independently determined the tumor thickness, Clark level of invasion, ulcerative state, and TMR for each lesion. Intraclass correlation coefficients and kappa scores for multiple ratings per subject were calculated.
Results: The intraclass correlation coefficients were 0.96 for tumor thickness and 0.76 for TMR. The kappa scores were 0.83 for ulcerative state and 0.60 for Clark level. These results indicated excellent agreement among the pathologists for measurements of tumor thickness, ulcerative state, and TMR and fair to good agreement for Clark level.
Conclusions: Appropriately trained and experienced histopathologists can assess prognostically important features of melanomas accurately and reproducibly. Given our recent finding of the significance of TMR in determining prognosis, it is important that this feature be assessed by a standardized method and documented for all primary cutaneous melanomas.

Training improves interobserver reliability for the diagnosis of scaphoid fracture displacement

Background The diagnosis of displacement in scaphoid fractures is notorious for poor interobserver reliability.

Questions/purposes We tested whether training can improve interobserver reliability and sensitivity, specificity, and accuracy for the diagnosis of scaphoid fracture displacement on radiographs and CT scans.

Methods Sixty-four orthopaedic surgeons rated a set of radiographs and CT scans of 10 displaced and 10 nondisplaced scaphoid fractures for the presence of displacement, using a web-based rating application. Before rating, observers were randomized to a training group (34 observers) and a nontraining group (30 observers). The training group received an online training module before the rating session, and the nontraining group did not. Interobserver reliability for training and nontraining was assessed by Siegel’s multirater kappa and the Z-test was used to test for significance.

Results There was a small, but significant difference in the interobserver reliability for displacement ratings in favor of the training group compared with the nontraining group. Ratings of radiographs and CT scans combined resulted in moderate agreement for both groups. The average sensitivity, specificity, and accuracy of diagnosing displacement of scaphoid fractures were, respectively, 83%, 85%, and 84% for the nontraining group and 87%, 86%, and 87% for the training group. Assuming a 5% prevalence of fracture displacement, the positive predictive value was 0.23 in the nontraining group and 0.25 in the training group. The negative predictive value was 0.99 in both groups.

Conclusions Our results suggest training can improve interobserver reliability and sensitivity, specificity and accuracy for the diagnosis of scaphoid fracture displacement, but the improvements are slight. These findings are encouraging for future research regarding interobserver variation and how to reduce it further.